Using Quasirandom Numbers in Neural Networks

Peter G. Anderson Roger S. Gaborski Ming Ge Sanjay Raghavendra Mei-Ling Lung Abstract We present a novel training algorithm for a feed forward neural network with a single hidden layer of nodes (i.e., two layers of connection weights). Our algorithm is capable of training networks for hard problems, such as the classic two-spirals problem. The weights in the rst layer are determined using a quasirandom number generator. These weights are frozen|they are never modi ed during the training process. The second layer of weights is trained as a simple linear discriminator using methods such as the pseudoinverse, with possible iterations. We also study the problem of reducing the hidden layer: pruning low-weight nodes and a genetic algorithm search for good subsets. Neural Network Structure Our goal is to construct feed-forward neural networks, such as Fig. 1, which can solve low dimensional twoclass discrimination problems, such as the famous twospirals problem [5]. Solving means that the network must learn the training set and generalize (interpolate and extrapolate) that learning in a reasonable way. The training data for the two-spirals problem, along with the generalization ability of one of our networks, is shown in Fig. 2. In most of our experiments, we use 200 points on each spiral, 400 training exemplars in all. The points lie in a 2a 2a square in R2 centered at the origin. We have experimented with various values of a in the range 1.0{4.0. A feed forward network to classify those points has three inputs, x0, x1, and x2. The pair (x1; x2) represents the input data to be classi ed, and x0 1 is a xed bias input to allow us to use zero as the threshold for the cells in the next layer. The single hidden layer consists of N nodes, denoted y = (y1; y2; : : : ; yN ). The output is the scalar, z. The transfer function of the Input nodes Output node